A Model-Based Approach to Measuring Expertise in Ranking Tasks
نویسندگان
چکیده
We apply a cognitive modeling approach to the problem of measuring expertise on rank ordering tasks. In these tasks, people must order a set of items in terms of a given criterion. Using a cognitive model of behavior on this task that allows for individual differences in knowledge, we are able to infer people’s expertise directly from the rankings they provide. We show that our model-based measure of expertise outperforms self-report measures, taken both before and after doing the task, in terms of correlation with the actual accuracy of the answers. Based on these results, we discuss the potential and limitations of using cognitive models in assessing expertise.
منابع مشابه
Investigating the Impact of Authors’ Rank in Bibliographic Networks on Expertise Retrieval
Background and Aim: this research investigates the impact of authors’ rank in Bibliographic networks on document-centered model of Expertise Retrieval. Its purpose is to find out what kind of authors’ ranking in bibliographic networks can improve the performance of document-centered model. Methodology: Current research is an experimental one. To operationalize research goals, a new test colle...
متن کاملInferring Expertise in Knowledge and Prediction Ranking Tasks
We apply a cognitive modeling approach to the problem of measuring expertise on rank ordering problems. In these problems, people must order a set of items in terms of a given criterion (e.g., ordering American holidays through the calendar year). Using a cognitive model of behavior on this problem that allows for individual differences in knowledge, we are able to infer people's expertise dire...
متن کاملLearning Models for Ranking Aggregates
Aggregate ranking tasks are those where documents are not the final ranking outcome, but instead an intermediary component. For instance, in expert search, a ranking of candidate persons with relevant expertise to a query is generated after consideration of a document ranking. Many models exist for aggregate ranking tasks, however obtaining an effective and robust setting for different aggregat...
متن کاملMeasuring the efficiency of a three-stage network using data envelopment analysis approach considering dual boundary
This paper presents a method for performance evaluation, ranking and clustering based on the double-frontier view to analyze the complex networks. The model allows us to open the structure of the “black box” and can help to obtain important information about efficient and inefficient points of the system. In this paper, we consider a three-stage network, in respect to the additional desirable a...
متن کاملA new approach based on data envelopment analysis with double frontiers for ranking the discovered rules from data mining
Data envelopment analysis (DEA) is a relatively new data oriented approach to evaluate performance of a set of peer entities called decision-making units (DMUs) that convert multiple inputs into multiple outputs. Within a relative limited period, DEA has been converted into a strong quantitative and analytical tool to measure and evaluate performance. In an article written by Toloo et al. (2009...
متن کامل